MASSACHUSETTS INSTITUTE OF TECHNOLOGY 
ARTIFICIAL INTELLIGENCE LABORATORY 


A.I. Memo No, 631 


May, 1981 


Color Vision and Image Intensities: 
When are Changes Material? 

John M. Rubin & W. A. Richards 


ABSTRACT 

Marr has emphasized die difficulty in understanding a biological system or its components without 
some idea of its goals. In this paper, a preliminary goal for color vision is proposed and analyzed. That 
goal is to determine where changes of material occur in a scene (using only spectral information)! This 
goal is challenging for two reasons. First, the effects of many processes (shadowing, shading from sur¬ 
face orientation changes, highlights, variations in pigment density) are confounded with the effects of 
material changes in the available image intensities. Second, material changes arc essentially arbitrary. 
We are consequently led to a strategy of rejecting the presence of such confounding processes. We 
show there is a unique condition, the spectral crosspoint, that allows rejection of die hypothesis that 
measured image intensities arise from one of the confounding processes. (If plots are made of image 
intensity versus wavelength from two image regions, and the plots intersect, we say that there is a 
spectral crosspoint.) We restrict our attention to image intensities measured from regions on opposite 
sides of an edge because material changes almost always cause edges. Also, by restricting our attention 
to luminance discontinuities, we can avoid peculiar conspiracies of confounding processes that might 
mimic a material chage. Our crosspoint conjecture is that biological visual systems interpret spectral 
crosspoints across edges as material changes. A circularly symmetric operator is designed to detect 
crosspoints; it turns out to resemble the double-opponent cell which is commonplace in biological 
color vision systems. . 
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FINDING MATERIAL CHANGES WITH COLOR 



Figure 1 . Notation. A) X and Y are two regions in an image. B) The available 
information is continuous functions of image intensity versus wavelength, / x (X) and 
/y(X), from regions X and Y respectively. C) Discrete representation of this information 
is possible by taking a number (here, three) of spectral samples (narrowband in this 
illustration, at wavelengths Xi, X2, and X3). 

Our goal is thus similar to the one first proposed by Land (1977), but more modest. Land was 
concerned, as we are, with computing information about surface properties of materials in a scene. 
His objective was to determine the reflectance of regions using only the available intensity informa¬ 
tion. Land reached his objective by assuming a greatly simplified Mondrian world—a flat, shadowless 
world composed of regions of uniform reflectance. We prefer to deal with the natural world in its 
entirety without unnecessary simplification, and instead limit our objectives. In this way we can 
exploit the regularities in the world as assumptions or constraints in the solution of our problem. The 
simple goal we will analyze, to reiterate, is just to determine when changes in an image arise from 
one material’s bordering (or occluding) another in a scene. We shall sec below that the problem to be 
solved in achieving our modest goal is still formidable. 
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glossaryTof symbols 

I. Processes 

M —material changes 

5— shadow (point source) 

^—highlight or gloss 

6 — change in surface orientation 
change in pigment density 

^pen—shadow-with penumbra (extended source) 

^mpfe—monotonic change in surface orientation 

II. Quantities measured b Image 

7x(A)—the continuous function of image intensity versus wavelength measured in a region X in an 
image. 

JiX —a discrete spectral sample at wavelength X* 

III. Continuous Scene Variables 

p(X)—the albedo or spectral reflectance function of a material. 

£o(X)—the diffuse component of the illumination. 

£y X)—the synthetic component of the illumination. 

J?(X)—the illuminant taken as a whole (when components are irrelevant), 
o—the penumbra constant. 

7 x—the thickness of pigmentation in region X. 

IV. Discrete Scene Variables 

If some function of wavelength in region X , then Q iX is a discrete sample of that 

function, at wavelength X„ from region X. 

Table 1. A glossary of symbols. 


Tabic 1. A glossary of symbols. 

2 The Scope of the Problem 

* 

The problem in achieving the proposed goal of finding changes of material (hereafter denoted by 
the symbol M 1 ) is that any given image intensity can arise in many different ways, depending on 

t 

the particular processes in effect. Thus given a single intensity value in the image, it is generally not 

possible to decide which of the many possible events in the world produced it. It can be said that the 

» * 

act of imaging a scene, or projecting it into two dimensions, confounds the effects of material changes, 
our interest here, with the effects of other processes. These confounding events include shadows, 
surface orientation changes, highlights, and variations in pigment density. Furthermore, the quest 

A 

for recovering M (material) changes from image intensities must succeed under a range of spectrally 
different iiluminants. A system that only worked properly given an illuminant having the spectral 
composition of the mid-day sun would be very limited. At dawn and dusk, when sunlight reddens, the 
system might fail. 

'The circumflex acccm is used in ihis paper lo denote abstract processes: it is intended to keep discussion about processes 
distinct from talk about simple variables or functions. 
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We can now summarize the problem that early color vision faces, if its goal is to discover material 
changes: 

The problem in determining where changes of material occur in a scene is that in the available image 
intensity values, the effects of many processes may be confounded. 

In the remainder of section 2, a notational scheme will be developed. In section 3, a general outline 
of the solution will be presented. 

Notation. A notational scheme is shown in figure 1. Regions in the static image will be denoted by 
X and Y. (The time variable will be ignored.) These letters will also be used to refer to regions in a 
scene which are the inverse projections of the image regions. The context will make clear the correct 
referent. The image intensity measurable in region X (or Y), say, as a function of wavelength, will be 
denoted by the function Ix{X) (or Iy(X)). Note that Ix[X) (or Iy(X)) represents all the information 
available from region X: it is the continuous spectral distribution of image intensity from X (or Y), as 
shown in the middle graph in figure 1. In section 5, we will refer to discrete approximations of Jx(X), 
which can be generated by sampling the image intensity from X at spectral points (Xj,..., \ n ), as 
shown in the bottom graphs in figure 1. In discrete computations, hx will denote the image intensity 
measured from region X at Xj. 

Since many symbols are used in this discussion, a glossary is presented in table 1. 


3 The Theme of the Solution: A Negative View 

3.1 The Strategy 

Given two spectral energy distributions I x (X) and Iy[X), how might we determine whether they 
arise from an M (material) change? Little can be said about the spectral nature of M changes; they 
are essentially arbitrary. No simple equations can relate image intensities from two regions composed 
of different materials. The major confounding processes—shadows (hereafter denoted 5), highlights 

A A A 

(H), surface orientation changes (O), and changes in pigment density (P)—however, produce lawful 
changes in the image. Simple equations can capture this lawfulness, as will be seen in section 4. 
Suppose that by examining image intensities from two regions we could eliminate the possibility that 
they arose from cither an S, H, 6 , or P change. Since we arc assuming die illuminant to be spectrally 
invariant in a neighborhood, we would like to conclude that the change was due to a material change. 
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A preliminary conjecture is thus proposed: 

When a difference between the image intensities 7*(X) and Iy(K) (taken from two image regions X 
and Y) does not arise from one of the confounding processes S, H, O, orP, then this difference between 

A 

X and Y is due to a difference in materials, or M. 

The conjecture above suggests a computational strategy of attempting to reject measurements of 
image intensities as arising solely from shadow (S), highlight (H), surface orientation change (O), or 
pigment density change (P), the lawful confounding processes in a scene. Our rejection strategy will 
be correct only if there are no confounding processes other than the ones mentioned. But, as will be 
discussed later, if in the course of rejecting the presence of S, H, O, and P, we also reject the presence 
of any of a large class of other possible confounding processes, the strategy will be a powerful one and 
the conjecture will be useful for practical purposes. 

It is important to note that rejecting the presence of a lawful process is often much easier than ac¬ 
cepting it. As mentioned above, lawful processes are associated with equations. These equations relate 
quantities measurable in the image (constants in the equations) to scene properties (variables in the 
equations) which are not directly measurable. Typically, these scene property variables (reflectances, 
for example) are constrained to take values within a certain interval. As will be seen below, rather 
than attempting to solve a system of equations (subject to constraints on the values of variables), it 
is often computationally simpler to determine whether the system (with constraints) can be solved. 

In this sense, rejecting a solution (process) is easier than accepting one. By analogy, disproving a 
conjecture about number theory (some universally quantified equation or inequality) with a single 
counterexample is simpler than demonstrating the theoremhood of the conjecture. 2 

Here’s a simple example. Consider the equations I\ — JK and / 2 — J K > where J and K are 
variables, and 7j and / 2 are constants. If the variables J and K are constrained to have values greater 
than unity, then a simple test can be made that might determine that the system has no solution. 
Specifically, if min(/i,/ 2 ) < 1, then there is no solution that obeys the constraints 3 . Intuitively, 

2 Of course, rejecting the occurrence of a process or event is logically equivalent to accepting the occurrence of its 
negation. But usually either a process or its complement, and not both, can be characterized mathematically. So the 
logical equivalence breaks down in favor of practical considerations, such as describability. Intuitively, rejecting the 
presence of a shadow is child's play compared to accepting the presence of a nonshadow. Mow could the class of all 
(visually interesting) events-that-are-not-shadows possibly be characterized? 'Shat class is certainly a peculiar collection 
of odds and ends, including such diverse members as paths of fireflies and holes in the ground. Visual systems "don’t 
care" if they're accepting processes or rejecting negations of processes; it only matters to us when we characterize what 
we think the system might be doing. 

3 Clcarl> the product of two numbers each greater than unity is greater than unity. And a number greater than unity 
raised to a power greater than unity is also greater than unity. The rejection condition min (I u I 2 ) < 1 is just the 
contrapositive of the two statements above. 





RUBIN & RICHARDS 6 FINDING MATERIAL CHANGES WITH COLOR 

* 

this decision about unsolvability is computationally easier than actually finding a solution. 


3.2 The Strategy Applied to some Confounding Processes 

Figure 2 illustrates intuitively how the rejection strategy will be applied. Each of the first three 

AAA 

panels (O, H, P ) show the effects (in a graph of image intensity versus wavelength) of a surface 
orientation change, a highlight, or a change in pigment density in a planar patch of a single material. 

A 

(Shadow changes S are similar to highlights.) As will be shown below, the effect of a surface orien¬ 
tation change is to cut down the direct illumination by a constant fraction. Highlight or gloss is a 
situation of increased image intensity at all wavelength. (A shadow is the opposite.) Finally, pigment 
density changes, such as the variations seen in the grain of wood, can be characterized as several light¬ 
absorbing filters in sequence. The effect of a stack of filters is to reduce the available light according 
to a power relation. (The concentration of a dye dispersed in liquid affects transmitted light according 
to a power law more precisely.) What all these natural processes have in common is that they act 
to increase or decrease image intensities across wavelength. Violation of those sorts of displacements 
can be used to reject the presence of shadow, orientation change, gloss or highlight, or change in 
pigment density. Therefore, if two spectral functions of image intensities are not related by one always 
lying above the other, then the functions come from two regions that might be composed of different 
materials. The lower right portion of figure 2 illustrates such a situation, which is a candidate for a 
material change. 

In the next section we examine simple models of the confounding processes. In section 5, we 

address the problems: How many channels docs a visual system require in order to reject the presence 

* 

of particular processes in a scene? What sort of computations with available image intensities allow 
rejection of confounding processes? 

* 

t 

4 The Physics Behind the Scenes 

4.1 The Image Intensity Equation 

When light is reflected from a surface into the eye, the image intensity /(X) depends on several 
factors. The surface properties of the object interact with the geometry of the viewing situation and 
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Figure 1 Three different relations are illustrated that can hold between two spectral functions of images 
intensities from regions X and Y. At top (left), a multiplicative relation (surface orientation change) is 
illustrated. At right is depicted an augmentative relation (highlight, for example). Bottom (left) is a power 
relation, typical of pigment density changes. At right is a change that is not an increase or decrease of intensity 
across wavelength. Such a situation is a possible material change. 

the spectral nature of the light source to produce image intensity. 4 In the case of a matte surface 5 , 
these effects combine multiplicatively to yield the following image intensity equation: 


I{\) = p(X)E(\)[N • L]R(0,<t>) (1) 

where p{\) is the reflectance or albedo of the surface, R{0,4>) is the bidirectional reflectance distribu¬ 
tion function (see Horn & Sjoberg, 1979) which describes properties of the surface dependent on its 
orientation with respect to viewer {<j>) and light source (0), and [N • L] is the angular relation between 
the surface normal, N, and the illuminant direction L. Any collection of multiple light sources is 

4 technically, /(X) for a point x,y on the surface is called image irradiance. Image intensity takes into account other 
factors such as pupil size and luminance constants. These details are not important to the argument here, and will be 
omitted. 

5 For specular surfaces, a different image intensity equation holds. See Appendix I. 
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proportion of the source can reach the object B) An (achromatic) image intensity profile is shown for the 
surface. 

equivalent to a single source (called here the "synthetic source") on unshadowed portions of surfaces 
(Silver, 1980), so regardless of the complexity of the illumination, a single function E(h), together 
with directions 0 and <f>, will characterize the direction and spectral nature of the illuminant for 
unshadowed surfaces. 

The effect of many natural processes that affect images can usually be characterized simply as ac¬ 
ting on one or more of the multiplicative factors in equation (1). For example, a shadow corresponds 
principally to a reduction of tire illuminant E(\). Surface orientation changes correspond to to 
changes in the [N • L] term, and pigment density changes affect only the p(X) term. (Highlight or gloss 
requires that a specular term be added to equation (1).) All of the processes discussed above can occur 
in a region of a single material. 

We now proceed to examine how common natural processes other than material changes will affect 
the image intensity equation, and what these confounding processes have in common that allows their 
rejection en masse. 
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4.2 Shadows 

A 

Suppose a shadow ( S ) falls on a surface composed of a single material, and furthermore, suppose 
that the only changes across the surface are these due to shadow. (Specifically, there will be no 
changes in surface orientation or variations in pigment density on the surface.) Then the shadow can 
be described by the following equations: 

IhM = [EsM + E D (k)]p(\) 

IshadeW = £d(X)p(X) ( j 

where p(X) is the albedo of the material. The image intensities, as functions of wavelength, from the 
lit and shadowed regions are Iut{X) and IshadeW, respectively. £/j(A) represents a diffuse component 
of illumination striking both lit and shadowed regions, and £s(X) is some additional synthetic source 
striking only the lit region. 6 

If the illuminant is an extended source, like the sun, there will be penumbrae (see figure 3), and a 
third equation must be included: 

/ pe „(A) = {aEsW + £b(X)MX) (26) 

where a is a constant between 0 and 1, and / pe „(A) is the image intensity (as a function of wavelength) 
in some region in the penumbra. (The symbol S pen will denote shadows with penumbrae.) 


4.3 Surface Orientation Changes 


Suppose that two image regions X and Y differ only in their surface orientations. That is, assume 
that X and Y receive the same illumination (no self-shading) and are composed of the same material. 
This process as been dentoed by the symbol 6. The patches X and Y that differ only in surface 


orientation have respective surface normals Nx and N> which form angles Ox and Oy with the 


6 If light is available from the shadowed region, it arises from some collection of sources. According to Silver (1980), a 
collection of sources is equivalent to a single source provided that they all illuminate the same surface patch. We call 
this synthetic source Ep(\). Furthermore, this diffuse synthetic source acts as if it lies in the direction of the viewer’s 
eye (Horn, 1975). Next consider the lit region. Again, by Silver’s proof, there is a single source that is equivalent to the 
illumination reaching the lit region, let’s call this source £' s *(\), and its direction L A .. Since we can see the lit region, 
we know that there is some component of L^. in the direction of the viewer. So we can perform a simple vector 
decomposition of the synthetic illuminant for the lit region into a component in the viewers direction (so this component 
is identical to the synthetic diffuse illuminant E I} (K) above), and some other component E s (\) with a new direction 
L.s-. So in equations ( 2 a) /w>( A) abbreviates a product of a term representing the spectral flux of the illuminant, a 


bidirectional reflectance term, and a surface orientation term, with the illuminant direction L in the viewer’s direction, 
likewise, F.s(X) stands for the product of of the same three terms above, but in some synthetic direction L 5 . 
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(synthetic) light source direction L, and angles <f> x and 4>y with the viewer direction V. Using the 
image intensity equation (1), the expected image’intensities from regions X and Y differing only in 
surface orientation are: 

MX) = P(X)£(X)[ N X • L]R(6 X> <t> x ) 

Iy{\) = p(X)£(X)[N K • L ]R(0y, <j>y) (3 ° J 

where again MX) and /y(X) are the image intensities from regions X and Y as functions of 
wavelength. 

Both equations of (3a) have a pair of multiplicative terms not involving wavelength. Let (3 X = 
[N* • L]ft(0x, 4> x ), and let j3y be similarly defined. Equations (3a) can now be simplified with this 
consolidation of constant factors: 


MX) = /MX)£(X) 

MX) = /?rp(XMX) 





4.4 Changes in Pigment Density 

The reflectance of materials is determined primarily by the density of a pigment in some embed¬ 
ding layer of the material, and the thickness of the pigment layer. The embedding layer in leaves, 
for example, is cellulose, and the usual pigment is chlorophyll. Some surfaces may be unevenly 
pigmented. The grain of a wooden table provides a good example of a change in pigment density 
across a surface. This sort of change will be labeled P. 7 

Several different laws relate changes in pigment density to changes in reflectance. (See Judd & 
Wyszecki, 1963.) Kubclka and Munk (1931) formulated a law that deals with the thickness and 
density of pigment on a solid background. Beer’s law describes the effect of concentration of a dye 
dispersed in a liquid on the transmittance of the liquid, or equivalently, the thickness of a scries of 
transmitting filters. The laws all differ in their details, yet there are some themes common to both. 

The pigmentation laws describe a smooth transition from an unpigmented state to a state of saturated 

* 

pigmentation. (Often, the unpigmented materials are white or grey, and darken as pigment density 
increases. Caucasians are usually pale in the winter, and darken gradually during the summer if 
allowed to frequent the beach.) More can be said about die smooth transition. Loosely speaking, 

'ilic difference between pigment density changes and changes of material is important. P changes are changes in the 
density of a single pigment in a single sort of embedding material; A7 changes are changes between two different 
pigments and (or) two different embedding materials. 
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the shape of the albedo function is preserved by the pigment density change. Two properties of the 
change of the albedo function make the notion above more precise. First, for a material characterized 
by Kubelka-Munk analysis or Beer’s law, a change in pigment density affects the albedo function by 
either strictly increasing or decreasing its value at all wavelengths. Second, two albedo functions of 
the same material (but having different pigment densities) are related monotonically 8 . The conditions 
above will be called the nornial pigmentation conditions , and pigmentation processes which obey them 
will be called normal 

Beer’s law (Wyszecki & Stiles, 1973) will serve as an example of a pigmentation law. Suppose X 
and Y are two regions that transmit light through stacks of identical filters. Suppose further, that X 
and Y have the same surface orientation, and differ only in the number of filters in the stack. Then the 
image intensities /\ (X) and Iy{ X) that we could expect to measure are: 


W =[p(A)p£*(X) 

/y-(A) =[p(X)]^£7*(X) 1 } 

where p(\) is the transmittance of a thin filter, and r/x and jy arc the number of such filters in regions 
X and Y, respectively. Since the viewer-object-illuminant geometry is identical in X and Y, the 
two multiplicative terms of the image intensity equation (1) that deal with scene geometry and do not 
appear in (4) can be considered "absorbed" as a constant factor in£'*(X). 

Two facts about equations (4) warrant attention. First, a power relation is described between the 
reflectances of two regions of different pigment thicknesses. This is the simplest such pigment relation, 
and can be considered as a base from which more complex laws develop. Second, note that regardless 
of the function p(X), the first normal pigmentation condition holds. That is, [p(X)]" 1 ' x > [p(A)]' Yy \ 
for all X, or vice versa. Since the illuminant and scene geometry are identical in regions X and 
Y , the normal pigmentation condition above is preserved in the available image intensities. Similar 
relationships between pairs of albedocs will hold for more complicated pigmentation laws (Kubelka & 
Munk, 1931), because such laws require monotonicity. 


4.5 Highlights 

The image intensity equation (1) only applies to matte surfaces. Highlight or specularity, a condi¬ 
tion when a surface acts as a partial mirror, is a common confounding process. In a highlighted region 

8 lwo single-valued functions / and <7 are related monotonkaly if f(r) > /( y) implies t/(.r) > y{y). In particular, two 
monotonically related functions have their local minima and maxima at the same values in their domain. 
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image intensities are due to both a reflection of the light source, and a matte component due to the 
albedo of the underlying material. Appendix I shows that if most of the illumination of a region is 
due to the source that is reflected in the highlight, then the highlight H can be described as a process 
that strictly augments image intensities at all wavelengths. That is, if the normaI highlight condition 
holds, the image intensity measurable in a highlighted region is the same as that of a neighboring 
matte region plus some (always positive-valued) function of wavelength 9 . Note this is identical to the 
first of the normal pigmentation conditions. Highlight differs from pigment density change in that it is 
always a positive change, and it does not necessarily preserve the shape of the image intensity function 
of wavelength from a neighboring matte regions. 


5 Details of Process Rejection: Spatial and Spectral Samples 

Our computational strategy is to examine image regions and to attempt to reject them as arising 
from the action of a confounding process on a single material. How might rejections be made? If a 
process is lawful, as the major confounding processes are, then some system of equations characterizes 
the effects of the process. Given measurements of image intensities, if there is always a plausible 
solution 10 to the equations describing a process, then we can always mathematically interpret the 
image intensities as arising from that process. When we can always interpret image intensities as being 
due to a particular process, we can never reject the occurrence of that process. Whether or not there 
always exist plausible solutions depends on the number of samples being taken. In this section we 
examine the minimum information needed to be sometimes able to reject the presence of the major 
confounding processes. We will answer the questions: What is the minimum number of spatial and 
spectral samples needed to reject image intensities as arising from each of the confounding processes? 
How would increasing the number of samples increase the number of correct rejections? 

(Spectral samples will at first be assumed to be taken at a single wavelength. In Appendix II, it is 

9 

shown that essentially any sort of spectral sample preserves the results of the following sections, which 
are derived using narrowband samples.) 

9 Since the word ''highlight'' denotes a region of greater brightness than surrounding regions, the norma! highlight 
condition is reasonable and necessary to avoid mathematical anomalies such as "highlights" which are darker than 
surrounding regions. 

10 A plausible solution to a set of equations is one which assigns meaningful values to physical variables. For example, 
a solution which assigns a number greater than one to a variable representing a reflectance is not physically meaningful 
and must be discarded 
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5.1 Rejecting the S Hypothesis 

Impossibility of Rejecting S with One Spectral Sample. Suppose one narrowband 
spectral sample is taken at some wavelength Xj. Two image intensities can then be measured: hx and 
hr, the intensities at Xj in image regions X and Y (see fig. 2). Can we reject the possibility that the 
only difference between regions X and Y is due to a shadow? Since either X or Y can be lit, and the 
other in shade, there are two equations to examine. In more formal terms, we need to know if we can 
rule out the existence of a plausible solution to either of two discrete versions of equations (2a): 


hx = [E\d + Eis)pi 
h y = EidPi 


corresponding to X lit and Y shadowed, or 



hx = EidPi 

hr — {Eid + Eis)pi {5b) 

corresponding to X shadowed and Y lit. Note that pi is the albedo of the material at X i( and E IS and 

Eid are the strengths of the synthetic and diffuse components of the illuminant. 

» 

A shadow interpretation is some solution to equations (5a) or (5b) that assigns physically meaning¬ 
ful values to variables. If the measured values of hx and I\y can be shown to be inconsistent with 
both equations (5a) and (5b), then there canot be a shadow interpretation. On the other hand, if any 
values of hx and 7jy yield a plausible solution to the equations, then there will always be a shadow 

4 

interpretation. If there is always an S interpretation, rejection is impossible, and we must resort to a 
greater number of spectral samples. 

Intuitively, we are given two measurements of image intensity at only one wavelength. We are 
faced with the question: Is there any pair of image intensities that cannot be construed as arising from 
lit and shadowed portions of the same material? Given two measurements, one will be darker than the 
other. The darker region can be interpreted as being a shadowed continuation of the lighter region. So 
any pair of measurements can be construed as a shadow. Therefore, there can be no rejection of the 
shadow interpretation with only a single spectral sample. 


The casual argument above can be made rigorous. Is there ever a possibility that both equations 


(5a) and (5b) will have no solution? First we’ll consider solutions to equations (5a). One solution is 


given as follows: 
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E\D = 
Pi — 


S 

hr 


6 


E iS = 6( 


hx 

IlY 




where 6 is the parameter of the one-dimensional solution space of equations (5a). (Two equations 
in three variables almost always have a one-dimensional solution space.) This solution is acceptable 
or plausible as long as all the variables take positive values, and pi £ (0,1). The latter constraint is 
satisfied by restricting 6 to the interval {I lY , oo). Note that E lD and pi are always positive, since 6 is 
restricted as above, and the measurement I lY is positive. Therefore all variables will take on positive 
values if£i S is positive, or, equivalently, if I\x > h Y . So whenever> I lY , there is a plausible 
solution to (5a). By the symmetry of equations (5b), it is clear that whenever I\ Y > 7ix, equations 
(5b) will have a plausible solution. Therefore, there will always be a solution to either (5a) or (5b), 
regardless of the values of measured image intensities. So there is always a shadow interpretation 
for monochromatic measurements; the darker region can be construed as the shadowed one. Hence, 
monochromacy is inadequate to reject measured image intensities as arising from shadows 11 . 


Two Spectral Samples. 

Perhaps adding a second spectral sample will help. Will samples at two wavelengths from each of 
regions X and Y sometimes allow the rejection of the shadow hypothesis about the X-Y difference? 
Or will any sets of measurements always have a shadow interpretation, as in the monochromatic case? 

Intuitively, dichromacy should allow some rejections of the shadow hypothesis. Suppose that in the 
first spectral sample, region X is lighter than region Y. Then the same ought to hold for die second 
spectral sample if Y is to be construed as a shadowed continuadon of the same material that composes 
X. So if one region is lighter in the first spectral sample, but darker in the second spectral sample, we 
probably aren’t looking at a shadow. This follows from noting that in the case of shadow, both regions 
reflect a diffuse component of illuminadon, while the lit region has a synthetic component in addition. 

A more formal demonstration can be made, by rewriting the shadow equations (2a) in discrete 


J1 The claim that monochromacy does not allow shadow rejections only applies to the type of computations describee 
here. Of course there might be some type of achromatic computation, involving, say, texture, that would allow the 
rejection of shadow. 
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form for two spectral samples: 

hx — {E\d + E ls )pi 
hx — {Eid + Eis)pi 
hy = EidPi 
hy — E 2 DP 2 

corresponding to X lit and Y shadowed, or 

I\x — E lD pi 
hx = E 2 DP 2 
hy — ( Eid + Eis)pi 
hy — { E 2 D + E 2 s)p 2 




corresponding to X shadowed and Y lit. 

Once again, the left-hand side terms are the measured image intensities, and the right-hand side 
contains the variables for which we attempt to solve the equations. Given any four measurements hx, 
hy, hx, and hy, will there always be a plausible solution to either equations (7a) or (7b)? 

Consider first solutions to (7a). We have four equations in six unknowns, suggesting a two- 
parameter family of solutions. 


Below is a solution to equations (7a), parameterized by 6 and e: 


Eid = 6 


Eh D 


Pi 


hy 


P2 = 
Eis = 

E2S == 




The constraints of the values of the variables are once again that pi and p 2 E (0,1), and that all the 
variables take on positive values. So extending the results, of the previous section on a single spectral 
sample, it is clear that whenever both 7 1X > hy and hx > hy, there is a solution to equations (7a). 
Symmetrically, whenever both hy > hx and hy > hx, there is a solution to equations (7b). So 
there is a dichromatic rejection condition for shadows. If (hx > hy and hy > hx) or (7iy > hx 

and hx > hy), there will be no plausible solution to equations (7a), nor to (7b), and the presence of 
shadow can be rejected. 
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SPECTRAL CROSSPOINT 


IMAGE 



Figure 4. Rejection of 5 by crosspoint in plots from regions X and Y of image 
intensity versus wavelength. 


There is a simple geometric interpretation of the dichromatic rejection condition: S can be rejected 
if the graphs of image intensity versus wavelength for regions X and Y intersect, or have a crosspoint. 
Furthermore, it has just been proved that the crosspoint condition is unique and minimal. It is minimal 
in the sense that it involves two spatial and two spectral samples; no smaller number of samples 
would do. It is unique in that, given two spatial and spectral samples, only the crosspoint condition 
holding among the four measurements can accurately lead to a rejection of S. Whenever there is no 
crosspoint, a shadow interpretation, possibly wrong 12 , can be found. A final point. Given two line 
segments (two spectral samples at two spatial locations yields two line segments in our system of rep¬ 
resentation) there is only one topological property that they have: intersection (or non-intersection). 
Fortunately, intersection, or crosspoint, has been shown to be a physically interesting condition in 
this problem, as well as a topologically interesting one. Figure 4 illustrates the dichromatic rejection 
condition for 5. 

12 A wrong interpretation is one that assigns values lo physical variables that do not correspond to the actual values. 
Interpretations of sensory data are discussed in more detail in Richards et al t 1981. 
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n-chromacy. It is easy to extend the dichromacy result above to the general case of n spectral 
samples (Xj,..., X n ). Imagine taking pairs of these spectral samples and plotting a line segment from 
region X and another segment from region Y. If any pair of spectral samples yields a crosspoint (the 
line segments intersect), then S can be rejected for the X-Y change. More formally, we can reject 
S n-chromatically if there exist j ^ k such that the line segment from I jX to I k x intersects the line 
segment from Ij Y to 4y- Equivalently, we can reject S n-chromatically if there exist j k such that 
(Ijx — f?y)(4x — 4 y) < 0. 

5.3 Rejecting the 6 Hypothesis 


One Spectral Sample. The surface orientation change equations (3b) can be re-expressed as a 
single equation: 



fr(X) 

MX) 


Px 

Py 



Equation (9) describes a simple proportionality holding over wavelength for image intensities. 
Clearly, if only a single spectral sample is taken at each of X and Y, a trivial proportionality will hold 
between the measurements I IX and I lY , regardless of their values. (That is, for all measurements of 
image intensities, there are appropriate values of p x and Py so that a discrete version of equation (9) 
holds.) Hence a surface orientation change cannot be rejected monochromatically. 


Two Spectral Samples. Will a second spectral sample sometimes allow the rejection of image 
intensities as arising from a surface orientation change? And if so, under what conditions? Equation 
(9) can be rewritten in discrete form for two spectral samples: 


hx _ Px 
hy Py 
hx _ Px_ 
hy Py ■ 



It is clear (by combining equations (10)) that we can reject the 6 hypothesis whenever 





(11) 



RUBIN & RICHARDS 18 FINDING MATERIAL CHANGES WITH COLOR 

Note that the nonproportionality condition above is narrower than the crosspoint condition that we 
derived for shadows; a crosspoint allows the rejection of both shadow and surface orientation change 
interpretations of image intensities. 

5.4 Rejecting the P Hypothesis 

In the preceding two sections, relationships among image intensities (at two spectral samples) 
were derived such that the discovery of the relationship could be taken as evidence that certain con¬ 
founding processes were not occurring in the image. The spectral crosspoint was the less strict of 
the two criteria derived. In this section, a different tack is taken. First, it is taken as obvious that 
monochromacy is inadequate to reject the presence of P. Next, it will be pointed out that a spectral 
crosspoint is good evidence that the image intensities under consideration do not arise from a change 
in pigment density 13 . It suffices to show (normal) pigmentation does not produce crosspoints. But this 
follows immediately from the first of the normal pigmentation conditions. Specifically, equations (4) 
can be rewritten for two spectral samples as 


hj£ _p7 x 

hr PT 
hx _P 2 X 
h y Pl Y 



which implies (by the first normal pigmentation condition) that if ^ >1, then fa > 1, or vice 

versa. Therefore, if ^ > 1 and ^ < 1 (one of two possible spectral crosspoint conditions), then 
the presence of a normal pigmentation process can be rejected. 


5.5 Rejecting the H Hypothesis 


The presence of a spectral crosspoint can also be taken as evidence that no normal highlight is 
present. Since a (normal) highlighted region has greater image intensity at all wavelengths than a 
neighboring matte region, no spectral crosspoints can arise from such a pair of regions. (Also note that 
shadow changes are formally equivalent to highlights, in that lit and higlightcd regions correspond 
to striclty augmentative changes to image intensities in neighboring shadowed and matte regions, 
respectively.) Hence the spectral crosspoint allows the rejection of normal highlights. 

1 O 

More precisely, it will be shown a crosspoint implies either (not P) or (P, but not a normal pigmentation process). 






RUBIN & RICHARDS 


19 


FINDING MATERIAL CHANGES WITH COLOR 


5.6 Rejecting Special Cases of S, 6, and H 

•at 

Shadows with Penumbrae. In the event of an extended source, shadows with penumbrae 

A 

('S'pen) can be rejected monochromatically (see Appendix II), but it should be noted that 
monochromatic rejection of the penumbra case requires three independent spatial samples, in con¬ 
trast to the two spatial samples needed for dichromatic rejection of S' (point source). That is, for sharp 
(point-source—generated) shadows, two spatial and two spectral samples are the minimum needed 
for rejection. For shadows with penumbrae, three spatial samples at a single wavelength suffice for 
rejection. Nonmonotonicity of the discrete plot of image intensity versus postion in the image at any 
wavelength is sufficient for rejecting S pcn . (Nonmonotonicity is when three collinear spatial samples 
are neither strictly increasing or decreasing. This is identical to the condition that the center spatial 

sample of the collinear three has the greatest intensity, which cannot occur when the line passes 
through lit, penumbra, and shaded regions.) 

"Simple" surface orientation changes. A special class of 6 changes can be rejected with 
a single spectral sample taken over three spatial regions. Let O stmp / e be the subset of 6 changes 
involving no inflections. O a i mp [ € is similar to S pen in that both describe spatially monotonic processes. 
Clearly then, O a i mp i e can also be rejected monochromatically using three spatial regions with the 
nonmonotonicity test. (Sec Appendix II, which treats the shadow-with-penumbra case, S pen .) 

Highlights and Monchromacy. Consider three collinear samples passing through a normal 
highlight such that the center sample is taken in the highlight, and the end samples are taken in 
nearby matte regions of the same material. A luminance profile (at a single spectral sample) shaped 
like an upside-down vee can be expected for our piecewise linear representation. Therefore,' a 
monotonic luminance profile, or a vee-shaped profile, can allow the rejection of the possibility that the 

three samples correspond to matte-highlight-matte regions. ' 

* 

5.7 Summary of Rejecting Individual Confounding Processes 

In the sections above, it was shown that a single spectral sample was not sufficient (except in 
special cases involving three spatial samples) to reject image intensities as the effect of a single major 
confounding process. Two spcctial samples, however, were proven adequate to sometimes allow 
rejection of image intensities as the effect of a single process. I he strictest rejection criterion occurred 
with surface orientation changes; nonproportionality was shown sufficient for rejection of 6. The 
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broadest criterion was the spectral cosspoint, derived from the shadow equations. Since shadow and 
(normal) highlight are similar in being processes that allow any strictly augmentative change over 
wavelength, the spectral crosspoint also allows the rejection of highlight. Furthermore, the crosspoint 
was shown to be the unique and minimal rejection criterion for S. The crosspoint is a special case of 
nonproportionality, and hence allows the rejection of surface orientation changes as well. (And the 
crosspoint is more secure than the nonpropordonality condition in situations of noisy measurements 
of image intensity.) Changes in pigment density are strictly augmentative (or subtractive) changes 
over wavelength, and therefore cannot cause crosspoints. So the crossoint is sufficient to reject P. 
Since each of the major confounding processes S, H, 6, and P can be rejected dichromatically by the 

presence of a spectral crosspoint, the crosspoint criterion for rejection of single confounding processes 
is adopted. 

5.8 Rejecting Combinations of Processes 

The discussion above focused on what happens when a single confounding process occurs. But by 
inspecting any photograph, it is obvious that arbitrarily chosen neighboring regions of the image may 
depict elements of the scene which differ in pigment density, surface orientation, and shadowing. (In 
particular, joint occurrences of O and S are common. Architecture provides examples in which one 
face of a polyhedral structure shades another.) 

How can the analysis be extended to cover instances of combinations of processes'! Perhaps we 
can restrict our attention to some subset of neighboring image regions in which material-change- 
mimicking conspiracies among the confounding processes are unlikely. 

Which Neighboring Image Regions should be Examined?. It would be computationally 
exhausting to examine all possible neighboring image regions for material changes. And unnecessary 
as well. Material changes invariably produce with luminance discontinuities or edges in the image. If 
finding material changes is the goal being pursued, there’s no point in looking for them where they’re 
not going to be. That is, for the purpose of discovering material changes, "neighboring regions" 
should be taken to mean regions separated by edges. 

Can several confounding processes coincide at a single edge? As pointed out above, shadows and 
surface orientation changes can occur at a single edge. But aside from this case of self-shading, it 

is generally believed that at an edge, a single process predominates (Marr & Hildreth, 1980; Marr, 
1982). 
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(In the case of self-shading, no crosspoint can occur. By examining the shadow and surface orienta¬ 
tion change equations, it can be seen that a joint*occurrence of S and O is mathematically equivalent 

A A 

to an occurrence of S alone. And S changes cannot induce crosspoints. So self-shading cannot cause 
crosspoints.) 

5.9 Accepting M by Default 

When a crosspoint occurs across an edge, we know it does not arise from one of our confounding 
processes. If the reasoning above is correct, it is highly unlikely that such a crosspoint is caused by 
a peculiar conspiracy among S, H, 6, and P across an edge. It is worth asking if we have fully 
captured the range of possible confounding processes by our models of S , H, 6, and P. All that could 
be said was that a highlighted region had greater image intensity at all wavelengths than a nearby 
matte region of the same material. The same is true for lit versus shadowed regions. And a slightly 
more stringent condition held for pigment density changes. It is clear from the discussion above that 
any process that acts on a scene variable(s) (that is a function of wavelength) by strictly increasing or 
decreasing its value (at all wavelengths) can be rejected with the spectral crosspoint. Thus, when a 
spectral crosspoint occurs across an edge, it is almost always due to a material change. 

5.10 False Targets: Effectiveness of the Spectral Crosspoint in M Detection 

A word should be said about the theory of M detection presented here. If our characterization 
of confounding processes is accurate, most spectral crosspoints across edges will be M changes. 
Exceptions include instances of highlight and pigment density changes that aren’t normal and rare 
(measure zero) coincidences of confounding processes at edges. So in signal detection language, the 
false target rate of crosspoints in M detection is low; most crosspoints will be material changes. 

As for the hit rate, it need not be the case that most M changes cause crosspoints in the image. 
Intuitively, only about half of the material changes in a scene will cause crosspoints. This should 
not be disturbing. The crosspoint computation is extremely easy, and provides immediate strong 
assertions. Of course a full theory of M detection is likely to involve other spectral computations, as 
well as nonspectral (textural) computations. 
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6 Relation to Psychophysics and Neurophysiology 

The theory we have presented begins with the physics underlying several scene processes affecting 
image intensities. Simple analysis of equations describing the processes has shown the crosspoint 
is the unique criterion for rejecting each of the confounding processes, and thus a good tool for 
spectrally identifying material changes. But what has this to do with color vision? The implica¬ 
tion of the results above is our crosspoint conjecture that biological visual systems interpret spectral 
crosspoints as material changes. 

Psychophysical Evidence. An interesting fact about human color vision is that it becomes un¬ 
stable under isoluminance conditions (Evans, 1948). Boundaries defined only by a chromatic change 
are weak, colors from one side of such a boundary are likely to invade the other. Color vision seems 
to require luminance discontinuities for stability. It was argued (section 5.8) that the only place in an 
image it makes sense to seek material changes is at edges. If color vision were concerned with the 
detection of material changes, then it would be an efficient system only if its computational resources 
were employed at edges. Besides, spectral computations carried out in arbitrary neighboring regions 
would be difficult, if not impossible, since many confounding processes could be acting together be¬ 
tween the two regions. The number of hypotheses to be entertained would become combinatorically 

nasty (that is, all possible subsets of S, 11, O, and P), and the biological value of enduring such 
tedium is dubious. 

Number of Spectral Channels. Biological color vision systems are usually di- or trichromatic 
(Walls, 1942). Why are there no pcntachromatic systems, say? The theory suggests an answer. 
Dichromacy was shown sufficient to enable a system to make strong M assertions across edges. 
Adding a third spectral sample increases the chances of finding a crosspoint. That is, trichromacy 
seems to provide higher hit and lower miss rates in M detection than dichromacy. Why not add 
spectral channels ad infinitum ? The spectral reflectances of natural objects are almost always functions 
that change slowly over wavelength (Krinov, 1971). (There appear to be rarely more than three 
extremal points in reflectance function of wavelength.) It s.ecms unlikely that a tetrachromatic system 
would detect enough additional crosspoints to be evolutionary advantageous. 

Comment on Physiology and Neurophysiology. 

frying to design an operator to detect spectral crosspoints provides some insights into 
neurophysiology. Although biological pholopigments are broadband (Dartnall,1962), Appendix III 
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symmetric operator that compares image intensities at a single spectral sample (here short wavelength, or S), 
over two spatial regions. Q The S+EL+ crosspoint detector. Two different Land units, S and L, are 
combined with a logical and across an edge, E. A nonzero response indicates a crosspoint. D) The S+EL+ 
unit resembles the double opponent unit See text. 

shows that the particular shape of the photopigments used for taking spectral samples, and their 

degree of overlap, have no effect on the basic crosspoint finding. 

The crosspoint (fig. 5a) is equivalent to the following inequality among image intensities: 

* 

{hx — hy){hx — hy) < 0 (13a) 

which is identical to 

{hx — h\')(—hx + hy) > 0 (136) 

Suppose wc desire a crosspoint operator that is circularly symmetric (for tessellation efficiency, say). 

Let the photopigments be S and L, having their respective \„ UI ’s at short and long wavelengths (sec 
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fig. 5a). Equation (13b) can be rewritten: 


(<Sx Sy)[—Lx + Ly) > 0 (13c) 

where S x denotes image intensity in region X viewed through photopigmentS, and so on. 

The operator depicted in fig. 5b is ideal for computing the ( S x — S Y ) factor in equation (13c). 
This sort of unit we call a Land unit" since it was suggested by Land’s (1977) retinex theory of color 
vision. The Land unit has zero output in a homogeneous field, and maximum output when the center 
receives more energy at short wavelengths than the surround. 

Obviously, the firing of a single Land unit, even a maximal firing, does not provide information 
about the presence or absence of a crosspoint. A single unit can only inform about events at single 
spectral sample. Consider, however, the nonzero firing of a pair of Land units across an edge, as 
shown in fig. 5c. The S Land unit’s nonzero firing implies S x > Sy', the L Land unit’s firing implies 
Ly > L x . The conjunction, or logical anding, of the two conditions over and edge provides a test 
for crosspoint. We propose, then, a crosspoint detector composed of the conjunction of two (spectrally 
different) Land units on opposite sides of an edge. Note that without the requirement of an edge, this 
unit might respond to gradations of image intensities due to the effects of confounding processes. We 
call this operator an S+EL+ unit 14 . 

Two further points should be raised about crosspoint detection with the SEL operator. First, the 
unit shown in fig. 5c only detects the type of crosspoint shown in fig. 5a. If Land units arc constrained 
to have only positive response, as neurons are, a second type of uniL L+ES+, would be required to 
detect the other type of crosspoint (Ii X < I\ y). 

Second, a positive response of an on-center off-surround Land unit on one side of an edge implies 
a positive response of an off-center on-surround unit on the other side of the edge. In fig. 5c, imagine 
replacing the L+ unit on the Y side of the edge with an L~ unit (off-center) on the X side. Suppose 
the L unit and an S "L unit, both on tire X side of an edge, were logically anded in a single unit. 
Such a unit might be sketched as in fig. 5d; it resembles a double-opponent cell, found in tire retina 
of goldfish and in the primary visual cortex of primates (Daw, 1972) 15 . A final point should be 

14 This unit can be compared to Marr and Ullman’s (1981) S+TS~ unit for directional selectivity. The two units are 
similar in that they have three components, the outer two of which are "polar" forms of the same computation (S+ 

and S differ in sign of contrast: S - ^ and L~^~ are different spectral samples. Also, the center components of the two 
units perform more sophisticated computations than the outer components. 

,5 The unit depicted in fig. 5d may be misleading in suggesting an arithmetic sum of two land units, rather than a logical 
anding. Onh detailed Quantitative neurophysiological study of such double-opponent units can reveal the computation 
they perform. 
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made about the double-opponent operator. It does not produce maximum output when it precisely 
straddles a edge with spectral crosspoint. The operator’s largest response occurs when it is slightly 
offset from the material change boundary. This is reminiscent ofproperites of the V 2 G convolution 
operator 16 for luminance discontinuities (Marr & Hildreth, 1980). 


7 Conclusion 

Color vision systems evolved to solve certain problems in making sense of natural images. Natural 
images are complicated things, however, caused by a myriad of processes. While first intuition might 
be to study color vision over a simplified or restricted domain, we feel much can be understood by 
considering the more complicated natural domain. Certain regularities emerge from the confusion of 
the world, and these lawful relationships can be exploited as assumptions in the solution of sensory 
problems. Even given assumptions about the physics underlying scenes, natural images are still com¬ 
plicated. While ambitious goals can be successfully pursued in limited domains (Land, 1977), we 
feel that generally, more modest goals are appropriate to complicated domains. Our simple goal is to 
detect changes of material in an image using spectral information. 

One unusual characteristic of material changes is that they are unconstrained. Since an uncon¬ 
strained process cannot be sought directly (for what equations could we seek solutions?), we resort to 
inferring material changes by rejecting a class of processes that confound spectral image intensities. 
Rejecting the presence of a lawful process, it was pointed out, is often computationally much simpler 

then solving explicitly for the variables of the process. 

* 

First, a computation based on a single spectral sample was sought that would allow, at least on 
some occasions, the rejection of the presence of one of the major confounding processes. But it 
was shown that no such computation exists (of the type discussed here); all neighboring pairs of 
regions look like lit and shaded portions of the same surface when viewed monochromatically. (It was 
shown, however, that special classes of confounding processes S pen and O simple , as well as H, could 
be rejected monochromatically using three spatial regions.) Next, it was found that adding a second 
spectral sample (for two spatial regions) did allow us to sometimes be able to reject the presence 
of a confounding processes. The spectral crosspoint was the broadest criterion derived, and it was 
shown that its discovery allowed the rejection of each of the confounding processes S, H, O, and P. 
(Furthermore, the spectral crosspoint was proved die unique and minimal criterion for the rejection 

le The laplacian of a two-dimensional gaussian. 
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A 

of 5.) It was argued that by restricting the search for spectral crosspoints to edges, that complications 
arising from the joint occurrence of several confounding processes were unlikely. (And if several 
confounding processes did occur coincidentally, a crosspoint could only be generated under very 
unusual circumstances of illumination.) The spectral crosspoint allows us to say with great confidence 
that image intensities do not arise from S, H, 6, P, or any of a large class of possible confounding 
processes. The simultaneous rejection of this large class of processes allows us to infer with great 
confidence that a material change is taking place. Our crosspoint conjecture is that biological visual 
sytems interpret crosspoints as material changes. 

When no crosspoint occurs, more sophisticated computations arc still possible (Richards, Rubin 
& Hoffman, 1981). The crosspoint strategy is not foolproof. While material assertions will be al¬ 
most always correct, there will be many missed material changes. Not all material changes produce 
spectral crosspoints. A good strategy for visual systems would be to locate the maximum absorption 
frequencies of their photopigments in order that a maximum number of crosspoints be detected 17 
. A study of the reflectances of natural objects could perhaps reveal if biological photopigments are 

located in wavelength in such a manner as to maximize the detection of crosspoints in an organism’s 
environment. 

Finally, it was shown that an operator for the detection of crosspoints, constrained to use only 
simple arithmetic functions, to output only positive values, and to be circularly symmetrical, is the 
S+EL+ unit, which performs a logical and of two spectrally different units across and edge. The 
SEL unit resembles the double-opponent cell commonly found in biological color vision systems. 

APPENDIX I 

* 

Details about Highlights 

The image intensity equation (1) does not apply to highlight. Highlight occurs on certain surfaces 
when two conditions hold. First, the viewer direction, surface normal, and illuminant direction must 
be approximately coplanar. Second, <f> ^ 6\ that is, the angle of incident illumination must be nearly 
equal to the angle of reflection to the viewer. When the two conditions above hold, certain materials 
will display mirrorlike qualities, and the illumination source will be imaged on the surface. Let 
EsourceO^) be the source that is imaged in the highlight. We can expect the following image intensities 

17 

Also, the photopigments should be located to minimize "doublecrossings,” or situations in which a pair of complementary 

and cancelling crosspoints occur between the spectral sample points. Such situations would cause material changes to be 
missed. 
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from two neighboring regions of the same surface orientation that differ only in that one region has a 
highlight: 


ImatteO 0 —(•E'source(A) -f- i?o(/ier(X))p(X) 

highlight ^0 ~^E aource {\) -}- (1 — S)(E aou rceO v ) + £ol/ier(A))/>(\) (15o) 

where 7 ma <t e (X) and Ihi g hii a ht{ X) are the image intensities expected from the matte and highlighted 
regions, and E otheT [ X) represents all illumination other than the source £ SO urce(X) reflected in the 
highlight. Note in the equation for the highlighted region that there are matte and specular com¬ 
ponents (Evans, 1948; Horn, 1977) in some linear combination determined by fraction 8. The specular 
component is just some coefficient multiplied by the source; the albedo plays no role. The matte 

component involves all illumination, not just the direct source, as well as the albedo p{\) of the 
material. 

Next, note the highlight equation can be rewritten as follows: 


highlight (M — 8L(\) -j- / mQ » e (X) (156) 

where L(\) — E source (\) [£!,ource(X) p(X). Therefore, the highlight becomes a purely 

additive process whenevei T( X) 0. We shall call this condition the normal highlight condition. It is 
equivalent to the following: 


JE 


'source 


Esource(1 0 ^ ^ 

W + E other {\) > 


for all X in the visually useful range. 



Since reflectances of surfaces usually have maxima around .7 (Krinov, 1971), the normal highlight 
criterion above is, loosely speaking, that the source provide a little more than twice die illumination 
than the diffuse light. Normal sunlit scenes will obey this criterion. 


APPENDIX II 

Shadows with Penumbrae ( S pen )\ Rejection with One Spectral Sample 


Here we seek to rule out the existence of a solution to cither of the following discrete equations: 

h,x = {E\s -\~Eio)pi 
h,Y — (aEis + E in )pi 
h,z = EioPi 


( 16 a) 
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Figure 6. Rejecting S pen by the nonmonotonicity of the discrete plot of image intensity 
versus spatial position at some wavelength. 


where X , Y and Z denote three collinear spatial regions being tested for their correspondence to lit, 
penumbra, and shaded regions, respectively, of a shadow with penumbra, or 

hx = E lDPl 

hr — [oEisE\o)pi (166) 

hz — (■Eis + Ei o) pi 

corresponding to the other case that Z is the lit region and X is shaded. 

A solution to equations (16a), parameterized by S is given below: 


Eid — S 







1 ) 


(17) 
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The constraints are that all the variables be positive, and a £ (0,1). E IS is positive if — 1 > 0 
Now the only interesting restriction is that a E (0,1). 

a>0 implies /i,y — h,z > 0 implies h, Y > h,z 
a<l implies h,x — h,z > h,Y — h,z implies h,x>h,Y 

Therefore, any violation of the restriction I ljX > h,y > h,z implies the unsolvability of equa¬ 
tions (16a). By the symmetry of the alternate equations (16b), it is clear they provide the restriction 
h,z > h,Y > hx • The pair of restrictions from (16a) and (16b) together imply that the three 
collinear image intensities must be either strictly increasing or strictly decreasing; that is, they must 

be monotonic. This rejection condition of nonmonotonicily is illustrated in figure 6, and can be 
summarized as follows: 

If the discrete plot of image intensity versus position across the S pen candidate is nonmonotonic, 
we can reject S pen . Equivalently, we can reject S pen if (I I X — I 1iY )(Ii,y — h,z) < 0. 

It is clear that with additional spectral samples, the nonmonotonicity of the discrete plot of image 
intensity versus wavelength at any wavelength is sufficient to reject S pen 

However, in practice, there is a problem with the type of operator suggested by the theory: at what 
scale should triplets of spatial regions be examined for nonmonotonicity? Shadows-with-pcnumbrae 
can occur with large variations in spatial extent in images. Should a whole range of triplets be ex¬ 
amined? The computational complexity of such a task should make one wary of it. We take this scale 
problem to imply that, for practical use, the crosspoint condition is preferred. 

APPENDIX III 

Broadband spectral samples cannot induce crosspoints. 

It will be shown that a crosspoint cannot arise in P, O, or S situations from an overlapping of 
samples. 

1] Assume the change from X to Y is due to one of the confounding processes. Therefore, the 
continuous spectral energy distributions 7*(X) and 7>(X) do not intersect. (Otherwise the proof in 
section 5.1 could be shown wrong by taking samples straddling the intersection.) Assume, without loss 
of generality, that 7* (X) is always greater than 7y(X). 

2] Assume that our two spectral samples will be measured by "photopigments" P x and P 2 where 
each Pi is simply a function mapping some wavelength interval into the unit interval (0,1). 
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3] The sample measured at X,- in region X, S iX , and in region Y, S iY , are defined as follows. 


Six = / 7x(X)P,(X) dX 

•/Xu 

SiY — I 7y(X)P;(X)dX 

* X*i 


(18) 


where Xj 2 > X t i and the interval (X^X^) is the range of wavelengths over which the 
"photopigment" P t is sensitive. 

4] Six > S iY , for all i, follows directly from the fact that 7*(X) > 7y(X), for all X. 

5] Therefore, spectral crosspoints cannot be induced by overlapping spectral samples. 
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